Reports loaded libraries in crashtracking reports#11000
Reports loaded libraries in crashtracking reports#11000gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits intomasterfrom
Conversation
bric3
added
type: enhancement
bric3
force-pushed
the
bdu/proc-memory-mappings
branch
from
168f306 to
0c5d63c
Compare
| }, | ||
| "files": { | ||
| "/proc/self/maps": [ | ||
| "55df6fe9d000-55df6fe9e000 r-xp 00000000 103:03 9818201 /home/bits/.sdkman/candidates/java/22.0.1-tem/bin/java", |
There was a problem hiding this comment.
I have a few questions:
- Should the path be scrubbed or removed?
- Are the additional details (POSIX attributes, date, size) actually useful?
- Is it necessary to report all segments, or would it be sufficient to include only those involved in the crashing stack frame, or strip them all given that the JVM already resolves the relative address within the library?
There was a problem hiding this comment.
- Should the path be scrubbed or removed?
It's unlikely to be needed, as these represents the memory mapping of mmaped libs, or executable, those are technical path, to JDK and other syslibs. Moreover, the file might have valuable version information in their filenames.
- Are the additional details (POSIX attributes, date, size) actually useful?
- Is it necessary to report all segments, or would it be sufficient to include only those involved in the crashing stack frame, or strip them all given that the JVM already resolves the relative address within the library?
I believe that the memory segment permissions are useful, but that's beside the point, the files event node of the RFC is about files and their content, here it's just the content of the /proc/self/maps (on Linux the JVM crash handler dumps that procfs file).
Note that the RFC doesn't state anything about scrubbing the mappings (Ruby and NodeJs already export /proc/self/maps). E.g.
ffff82b80000-ffff82b87000 r-xp 00000000 00:208 3892376489 /vendor/bundle/3.4.9/ruby/3.4.0/gems/io-event-1.14.0/lib/IO_Event.so
There was a problem hiding this comment.
I'll take a look a redacting paths in a later PR.
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 61 metrics, 10 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~cf001677e2, baseline=1.61.0-SNAPSHOT~b7fb6fc123
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.06 s) : 0, 1060110
Total [baseline] (11.095 s) : 0, 11095337
Agent [candidate] (1.068 s) : 0, 1068061
Total [candidate] (11.054 s) : 0, 11053693
section appsec
Agent [baseline] (1.252 s) : 0, 1252482
Total [baseline] (11.247 s) : 0, 11247364
Agent [candidate] (1.251 s) : 0, 1251430
Total [candidate] (11.13 s) : 0, 11129924
section iast
Agent [baseline] (1.234 s) : 0, 1233528
Total [baseline] (11.336 s) : 0, 11336207
Agent [candidate] (1.235 s) : 0, 1235411
Total [candidate] (11.368 s) : 0, 11367584
section profiling
Agent [baseline] (1.189 s) : 0, 1189416
Total [baseline] (11.09 s) : 0, 11089890
Agent [candidate] (1.187 s) : 0, 1187181
Total [candidate] (11.079 s) : 0, 11078818
gantt
title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~cf001677e2, baseline=1.61.0-SNAPSHOT~b7fb6fc123
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.224 ms) : 0, 1224
crashtracking [candidate] (1.197 ms) : 0, 1197
BytebuddyAgent [baseline] (629.555 ms) : 0, 629555
BytebuddyAgent [candidate] (633.542 ms) : 0, 633542
AgentMeter [baseline] (29.419 ms) : 0, 29419
AgentMeter [candidate] (29.875 ms) : 0, 29875
GlobalTracer [baseline] (257.747 ms) : 0, 257747
GlobalTracer [candidate] (259.956 ms) : 0, 259956
AppSec [baseline] (31.868 ms) : 0, 31868
AppSec [candidate] (32.334 ms) : 0, 32334
Debugger [baseline] (60.514 ms) : 0, 60514
Debugger [candidate] (61.079 ms) : 0, 61079
Remote Config [baseline] (635.818 µs) : 0, 636
Remote Config [candidate] (624.982 µs) : 0, 625
Telemetry [baseline] (7.988 ms) : 0, 7988
Telemetry [candidate] (8.864 ms) : 0, 8864
Flare Poller [baseline] (4.986 ms) : 0, 4986
Flare Poller [candidate] (4.33 ms) : 0, 4330
section appsec
crashtracking [baseline] (1.213 ms) : 0, 1213
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (661.387 ms) : 0, 661387
BytebuddyAgent [candidate] (661.3 ms) : 0, 661300
AgentMeter [baseline] (12.155 ms) : 0, 12155
AgentMeter [candidate] (12.247 ms) : 0, 12247
GlobalTracer [baseline] (259.22 ms) : 0, 259220
GlobalTracer [candidate] (258.648 ms) : 0, 258648
IAST [baseline] (24.238 ms) : 0, 24238
IAST [candidate] (24.101 ms) : 0, 24101
AppSec [baseline] (178.555 ms) : 0, 178555
AppSec [candidate] (178.194 ms) : 0, 178194
Debugger [baseline] (66.792 ms) : 0, 66792
Debugger [candidate] (66.569 ms) : 0, 66569
Remote Config [baseline] (655.704 µs) : 0, 656
Remote Config [candidate] (640.289 µs) : 0, 640
Telemetry [baseline] (8.34 ms) : 0, 8340
Telemetry [candidate] (8.425 ms) : 0, 8425
Flare Poller [baseline] (3.584 ms) : 0, 3584
Flare Poller [candidate] (3.648 ms) : 0, 3648
section iast
crashtracking [baseline] (1.229 ms) : 0, 1229
crashtracking [candidate] (1.205 ms) : 0, 1205
BytebuddyAgent [baseline] (799.684 ms) : 0, 799684
BytebuddyAgent [candidate] (800.548 ms) : 0, 800548
AgentMeter [baseline] (11.437 ms) : 0, 11437
AgentMeter [candidate] (11.528 ms) : 0, 11528
GlobalTracer [baseline] (248.712 ms) : 0, 248712
GlobalTracer [candidate] (249.599 ms) : 0, 249599
IAST [baseline] (25.474 ms) : 0, 25474
IAST [candidate] (25.62 ms) : 0, 25620
AppSec [baseline] (26.645 ms) : 0, 26645
AppSec [candidate] (26.632 ms) : 0, 26632
Debugger [baseline] (70.161 ms) : 0, 70161
Debugger [candidate] (69.484 ms) : 0, 69484
Remote Config [baseline] (540.952 µs) : 0, 541
Remote Config [candidate] (524.312 µs) : 0, 524
Telemetry [baseline] (9.764 ms) : 0, 9764
Telemetry [candidate] (10.477 ms) : 0, 10477
Flare Poller [baseline] (3.559 ms) : 0, 3559
Flare Poller [candidate] (3.579 ms) : 0, 3579
section profiling
crashtracking [baseline] (1.187 ms) : 0, 1187
crashtracking [candidate] (1.195 ms) : 0, 1195
BytebuddyAgent [baseline] (686.848 ms) : 0, 686848
BytebuddyAgent [candidate] (685.229 ms) : 0, 685229
AgentMeter [baseline] (9.053 ms) : 0, 9053
AgentMeter [candidate] (9.028 ms) : 0, 9028
GlobalTracer [baseline] (215.828 ms) : 0, 215828
GlobalTracer [candidate] (215.72 ms) : 0, 215720
AppSec [baseline] (32.662 ms) : 0, 32662
AppSec [candidate] (32.471 ms) : 0, 32471
Debugger [baseline] (66.589 ms) : 0, 66589
Debugger [candidate] (65.506 ms) : 0, 65506
Remote Config [baseline] (580.8 µs) : 0, 581
Remote Config [candidate] (590.679 µs) : 0, 591
Telemetry [baseline] (7.764 ms) : 0, 7764
Telemetry [candidate] (8.459 ms) : 0, 8459
Flare Poller [baseline] (3.519 ms) : 0, 3519
Flare Poller [candidate] (3.548 ms) : 0, 3548
ProfilingAgent [baseline] (94.163 ms) : 0, 94163
ProfilingAgent [candidate] (94.415 ms) : 0, 94415
Profiling [baseline] (94.714 ms) : 0, 94714
Profiling [candidate] (94.974 ms) : 0, 94974
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~cf001677e2, baseline=1.61.0-SNAPSHOT~b7fb6fc123
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1062738
Total [baseline] (8.908 s) : 0, 8908192
Agent [candidate] (1.06 s) : 0, 1059902
Total [candidate] (8.88 s) : 0, 8879758
section iast
Agent [baseline] (1.231 s) : 0, 1231001
Total [baseline] (9.613 s) : 0, 9613358
Agent [candidate] (1.231 s) : 0, 1231312
Total [candidate] (9.613 s) : 0, 9613227
gantt
title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~cf001677e2, baseline=1.61.0-SNAPSHOT~b7fb6fc123
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.211 ms) : 0, 1211
crashtracking [candidate] (1.203 ms) : 0, 1203
BytebuddyAgent [baseline] (631.412 ms) : 0, 631412
BytebuddyAgent [candidate] (631.16 ms) : 0, 631160
AgentMeter [baseline] (29.735 ms) : 0, 29735
AgentMeter [candidate] (29.358 ms) : 0, 29358
GlobalTracer [baseline] (259.645 ms) : 0, 259645
GlobalTracer [candidate] (257.54 ms) : 0, 257540
AppSec [baseline] (32.094 ms) : 0, 32094
AppSec [candidate] (31.864 ms) : 0, 31864
Debugger [baseline] (60.154 ms) : 0, 60154
Debugger [candidate] (59.75 ms) : 0, 59750
Remote Config [baseline] (609.999 µs) : 0, 610
Remote Config [candidate] (589.12 µs) : 0, 589
Telemetry [baseline] (8.164 ms) : 0, 8164
Telemetry [candidate] (8.081 ms) : 0, 8081
Flare Poller [baseline] (3.535 ms) : 0, 3535
Flare Poller [candidate] (4.194 ms) : 0, 4194
section iast
crashtracking [baseline] (1.214 ms) : 0, 1214
crashtracking [candidate] (1.205 ms) : 0, 1205
BytebuddyAgent [baseline] (799.343 ms) : 0, 799343
BytebuddyAgent [candidate] (799.017 ms) : 0, 799017
AgentMeter [baseline] (11.415 ms) : 0, 11415
AgentMeter [candidate] (11.415 ms) : 0, 11415
GlobalTracer [baseline] (248.001 ms) : 0, 248001
GlobalTracer [candidate] (248.063 ms) : 0, 248063
IAST [baseline] (25.372 ms) : 0, 25372
IAST [candidate] (25.478 ms) : 0, 25478
AppSec [baseline] (26.445 ms) : 0, 26445
AppSec [candidate] (27.553 ms) : 0, 27553
Debugger [baseline] (69.225 ms) : 0, 69225
Debugger [candidate] (68.071 ms) : 0, 68071
Remote Config [baseline] (532.33 µs) : 0, 532
Remote Config [candidate] (519.377 µs) : 0, 519
Telemetry [baseline] (9.645 ms) : 0, 9645
Telemetry [candidate] (10.207 ms) : 0, 10207
Flare Poller [baseline] (3.572 ms) : 0, 3572
Flare Poller [candidate] (3.658 ms) : 0, 3658
LoadParameters
See matching parameters
SummaryFound 7 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 16 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~cf001677e2, baseline=1.61.0-SNAPSHOT~b7fb6fc123
dateFormat X
axisFormat %s
section baseline
no_agent (1.236 ms) : 1225, 1248
. : milestone, 1236,
iast (3.485 ms) : 3435, 3536
. : milestone, 3485,
iast_FULL (6.344 ms) : 6277, 6412
. : milestone, 6344,
iast_GLOBAL (3.753 ms) : 3692, 3813
. : milestone, 3753,
profiling (2.346 ms) : 2323, 2369
. : milestone, 2346,
tracing (1.916 ms) : 1900, 1932
. : milestone, 1916,
section candidate
no_agent (1.219 ms) : 1208, 1229
. : milestone, 1219,
iast (3.251 ms) : 3206, 3296
. : milestone, 3251,
iast_FULL (5.983 ms) : 5923, 6044
. : milestone, 5983,
iast_GLOBAL (3.625 ms) : 3566, 3684
. : milestone, 3625,
profiling (2.094 ms) : 2076, 2113
. : milestone, 2094,
tracing (1.902 ms) : 1885, 1920
. : milestone, 1902,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~cf001677e2, baseline=1.61.0-SNAPSHOT~b7fb6fc123
dateFormat X
axisFormat %s
section baseline
no_agent (18.483 ms) : 18292, 18673
. : milestone, 18483,
appsec (19.143 ms) : 18950, 19336
. : milestone, 19143,
code_origins (18.154 ms) : 17975, 18332
. : milestone, 18154,
iast (17.818 ms) : 17642, 17993
. : milestone, 17818,
profiling (19.669 ms) : 19472, 19867
. : milestone, 19669,
tracing (18.634 ms) : 18447, 18820
. : milestone, 18634,
section candidate
no_agent (19.085 ms) : 18888, 19283
. : milestone, 19085,
appsec (18.721 ms) : 18535, 18907
. : milestone, 18721,
code_origins (18.125 ms) : 17946, 18303
. : milestone, 18125,
iast (17.761 ms) : 17585, 17938
. : milestone, 17761,
profiling (18.656 ms) : 18467, 18844
. : milestone, 18656,
tracing (17.946 ms) : 17768, 18125
. : milestone, 17946,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~cf001677e2, baseline=1.61.0-SNAPSHOT~b7fb6fc123
dateFormat X
axisFormat %s
section baseline
no_agent (1.498 ms) : 1486, 1510
. : milestone, 1498,
appsec (3.887 ms) : 3662, 4112
. : milestone, 3887,
iast (2.296 ms) : 2227, 2366
. : milestone, 2296,
iast_GLOBAL (2.331 ms) : 2261, 2402
. : milestone, 2331,
profiling (2.127 ms) : 2070, 2184
. : milestone, 2127,
tracing (2.117 ms) : 2062, 2171
. : milestone, 2117,
section candidate
no_agent (1.493 ms) : 1481, 1505
. : milestone, 1493,
appsec (3.844 ms) : 3619, 4070
. : milestone, 3844,
iast (2.291 ms) : 2221, 2361
. : milestone, 2291,
iast_GLOBAL (2.337 ms) : 2267, 2407
. : milestone, 2337,
profiling (2.128 ms) : 2071, 2185
. : milestone, 2128,
tracing (2.091 ms) : 2037, 2145
. : milestone, 2091,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~cf001677e2, baseline=1.61.0-SNAPSHOT~b7fb6fc123
dateFormat X
axisFormat %s
section baseline
no_agent (15.489 s) : 15489000, 15489000
. : milestone, 15489000,
appsec (14.809 s) : 14809000, 14809000
. : milestone, 14809000,
iast (17.935 s) : 17935000, 17935000
. : milestone, 17935000,
iast_GLOBAL (18.268 s) : 18268000, 18268000
. : milestone, 18268000,
profiling (14.803 s) : 14803000, 14803000
. : milestone, 14803000,
tracing (14.854 s) : 14854000, 14854000
. : milestone, 14854000,
section candidate
no_agent (15.416 s) : 15416000, 15416000
. : milestone, 15416000,
appsec (14.723 s) : 14723000, 14723000
. : milestone, 14723000,
iast (18.562 s) : 18562000, 18562000
. : milestone, 18562000,
iast_GLOBAL (18.077 s) : 18077000, 18077000
. : milestone, 18077000,
profiling (14.907 s) : 14907000, 14907000
. : milestone, 14907000,
tracing (15.016 s) : 15016000, 15016000
. : milestone, 15016000,
|
RFC: https://github.com/DataDog/libdatadog/blob/main/docs/RFCs/0011-crashtracker-structured-log-format-V1_X.md Note that on BSD / macOs the loaded libs is not coming from procfs.
bric3
force-pushed
the
bdu/proc-memory-mappings
branch
from
0c5d63c to
cf00167
Compare
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
There was an unexpected error while creating the working branch: cannot create branch DataDog/dd-trace-java/merge_request:{repository:{owner:"DataDog" name:"dd-trace-java"} queued_at:{seconds:1774963409 nanos:58994000} status_with_reason:STATUS_WITH_REASON_QUEUED_RETRY_CHECKS_FAILED priority:200 created_at:{seconds:1774963409 nanos:59806000} pull_request_number:11000 base_branch:"master" head_sha:"cf001677e24c487241a6d1789d4cca45a11570af" id:"3bae9e6c-5317-478f-a79c-82a9461df5ac" working_branch_sha:"fc715e30c649ca911ae425b602a2b7e43318edd5" gitlab_pipeline_id:105265050 temporal_execution_workflow_id:"019d440c-36b7-70da-ba45-578729a37583_344" temporal_execution_run_id:"019d4443-f8c3-79ba-afb4-b18f655dc017" status:STATUS_RETRYING triggered_by:"brice.dutheil@datadoghq.com" merge_status_workflow_id:"59ea87b6-1716-4b0d-8613-154a1c8ba3f0_47" ddci_request_id:"7429247253451350653"} name:"mq-working-branch-master-db060a6" temporal_execution_workflow_id:"019d447d-c868-7755-8dfb-5b9690752b68_122" temporal_execution_run_id:"019d447f-1b60-72b0-b4c3-f600792112e9" This could indicate that something doesn't work properly with the build system or that this one has reached its maximum capacity. DetailsError: cannot create branch DataDog/dd-trace-java/merge_request:{repository:{owner:"DataDog" name:"dd-trace-java"} queued_at:{seconds:1774963409 nanos:58994000} status_with_reason:STATUS_WITH_REASON_QUEUED_RETRY_CHECKS_FAILED priority:200 created_at:{seconds:1774963409 nanos:59806000} pull_request_number:11000 base_branch:"master" head_sha:"cf001677e24c487241a6d1789d4cca45a11570af" id:"3bae9e6c-5317-478f-a79c-82a9461df5ac" working_branch_sha:"fc715e30c649ca911ae425b602a2b7e43318edd5" gitlab_pipeline_id:105265050 temporal_execution_workflow_id:"019d440c-36b7-70da-ba45-578729a37583_344" temporal_execution_run_id:"019d4443-f8c3-79ba-afb4-b18f655dc017" status:STATUS_RETRYING triggered_by:"brice.dutheil@datadoghq.com" merge_status_workflow_id:"59ea87b6-1716-4b0d-8613-154a1c8ba3f0_47" ddci_request_id:"7429247253451350653"} name:"mq-working-branch-master-db060a6" temporal_execution_workflow_id:"019d447d-c868-7755-8dfb-5b9690752b68_122" temporal_execution_run_id:"019d447f-1b60-72b0-b4c3-f600792112e9" FullStacktrace: |
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
mergequeue build completed successfully, but the github api returned an error while merging the pr. DetailsError: PUT https://api.github.com/repos/DataDog/dd-trace-java/pulls/11000/merge: 405 Merge already in progress [] (Request ID: 8244:C5C4:19122F8:68038F2:69CC0EEA) FullStacktrace: |
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
3 participants
What Does This Do
Hotspot crashlogs have a Dynamic Libraries section, this PR reports it in the error event.
There are some differences between OSes, on Linux the JVM uses the procfs to write that section; on BSD, the JVM uses an internal track of loaded libraries, as such this data is named
dynamic_librariesMotivation
The dynamic libraries provide useful information to the actual path of the libraries, in particular the name of each lib can hint at the expected version of the native lib.
Additional Notes